1. Field of the Invention
The present invention relates to the field of image processing, computer vision and augmented processing technology, and in particular to the method for editing propagation of video and image content based on local feature structure preservation.
2. Background of the Invention
Editing based on the video and image contents is among the most common technology for image processing in the field of digital image processing. Example of editing based on the video and image contents may comprise, for example, changing color of the video and image (color transformation), merging objects from different video sources to form a video without editing traces (seamless cloning), extracting accurately hairs from an image (matting) or the like. For the art designers and video editors, it requires a lot of manual editions to edit the color and content of a video. In fact, there are some intrinsic relationships among content features of the video and image. In case that the video and image can be edited automatically according to these intrinsic relationships, it is possible to increase dramatically the efficiency for editing video and image.
Researches about image editing have been conducted widely. Zeev Farbman et al. proposed in 2010 an editing propagation method based on the diffusion map, in which the diffusion distance is used to measure affinity among all pixels. This method is neither efficient, nor can effectively reflect the non-affinity among pixels. In addition, this method is not appropriate for processing pixels in the color transition region.
As for color transformation, Eric Reinhard et al. in the University of Bristol firstly proposed a global method for color transformation in 2001. In this method, the target image and reference image are firstly converted from the RGB color space to the LAB color space. Then, the expectations and standard deviations along each axis of LAB color space are calculated, each pixel in the target image is scaled and shifted, and finally each pixel value is transformed back to the RGB color space. Although this method is simple and effective, the user is required to specify the reference for color transformation in case of a complex image.
As for cloning, Perez et al. proposed in 2003 a method for merging the scene and object based on the Poisson equation and the Dirichlet boundary conditions. Although the inserted object can be merged appropriately, this method consumes time and space. In 2009, Zeev Farbman et al. presented an image and video cloning method based on mean-value coordinates, which greatly improves the time and space consumption of the Poisson method. However, the mean-value cloning is susceptible to the shape of the inserted object.
Matting was firstly proposed by Jian Sun et al. in 2004. This method followed the principle of the Poisson equation to conduct the task of matting. However, this method suffers from low calculation speed and large consumption of storage space, and cannot extract the foreground well in the semi-transparent image region. Ahat Levin et al. proposed a spectral matting in 2008. Although this method improves to a certain extent the accuracy of matting, it still cannot extract the foreground well in the semi-transparent image region.
As for colorization of gray images, Welsh et al. firstly presented in 2001 a method for colorizing gray images based on gray matching. In this method, it is necessary to provide a color image which is similar to the scene of the gray image, and the gray image is colorized according to the gray matching between these two images. However, a gray image with complex scene cannot be colorized well by this method, and too much interaction may be involved during colorization.